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ABSTRACT 

yoale of the computation that extracts disparity from poirp 
of a scene are defined, and the constraints Imposed upon that 
by the Ihreo-di men»i one! structure o f the nor Id are 
E^presoing the computation as a grey-level correlation i< 
inadequate. A precise expression of the goal a of the 
is possible in 0 I on-level synbolic visual processori the 
translate In this environment to pre-requ i ei tea pn the 
blndlhB of disparity values to low-level ayhbols, The outline of a 
method baaed on thia ia given. 
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i n t rOdUC t i qn 


Commercial pressures have led to considerable interest In the 
automatic extraction ol disparity information from pairs el picture* <jf a 
scene. Since 13SS* there has been available a nachlna, the Hi Id-flaytheon 
BiS etereomat automated ptotler, uhich can dr an a tonloLr nap from two 
aerial photoyraphs* The machine correlates intensity measwr q-nentg 
Obtained over focal scans wade on the (bo images: the scan paths are the 
machine's current appro*imat ion to the contour lines [i.e. lines of 
■constant disparity!* and the failures ol correspondence -between the Acting 
on the tuc images 3rs used to improvo the approximation* Adequate 
accuracy, if achieved at all. la reached within about a in iterations* 

Machines that Sect, to assign disparity values to an Image Pyj 
performing correlation* between intensity arrgyg are subject to 
troiiblosohe problems due to focal minima! Hori, Kidode & Asada (1973) 
describe the problems, and have recently implemented some ways Of 
avoiding (hem* Their principal cutes are I i i to tbrrefate the tup images 
using local averages taken ovsr regions that are initially relatively 
large, anti which are subsequently reduced in jiao as the solution is 
approached: and (it) to avoid local winifia traps by introducing e ^tra I T 
amount of gaussian nplgg into the imeges. These techniques reduce 
consider ably the incidence of false assignments* but they fall to remove 
them altogether* 

There ham been less progress in the study of parallel algorithms 
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for fiaking uaa of di -spar Sty information, despite considerable recent 
intflruat in the procnseing of disparity information by the visual system, 
[Bar tow, BI akenore & Peitigreu {19673, Julesz I1B71)) ; JuIbbz 11971 
p204l , and Sparling (19701 have both- suggested possible Scheeas. Julesz'a 
mods | is informal in nature, being uore phenoneno logics I than 
capputat1dnalr it is wary interesting because despite its great 
simplicity, it displays an astonishing nwljsr of the properties that are 
exhibited by the human disparity processing system. Sperling's mode I le 
isonfl formal, but it ia difficult to tell hou uell it would work. This is 
3 problem w^th all cample* parallel methods; they are very expensive to 
simulate, and 11 is extremely difficult t& derive analytically, from* 
eyetoh with complex non-linear -coMponente, quantities that could be 
measured experimentally. About the best One can hope to do at present Is 
to state Criteria that distinguish one fahlly of nethods trei* another f 
and ask whether those criteria are satisfied by the particular method 
that we u-SOi 

This note enquires about the exact nature of the disparity 
Computation. ]t ie in same sense a Correlation, and because that is 
common knowledge and fairly precise, a deeper character f eat i on h3S not 
bfcpcn sought. Blrl In order to farnylate a method ol carrying it out, ona 
needs tC be very precise about the goals Of the computation, and about 

a 

the constraints imposed upon it by the structure of the thr«-dihension*I 
world. Unless one uses * nethod that is based on all and only th* correct 
assumptions, there Mill be situations in which it will fail 
urmpeessar i !y. 
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tluwuf Inp d i scar i- fij 

If ii scene is photographed Irom tup slightly el I f for ont positions, 
the relative positions of the objects in the scene ull I dltf^c slightly 
on the two i mage a. The <i I serepaneits of Intsrggt arise from the different 
distances of the Objects frou the viewing position, and measurements of 
the di acrepancies cwtaln useful information about the relative distances 
of tha objects. The tern bin ocular dJjp 3 ri,ty refers to the difference in 
the angle from each eye to a point in th* sbeho, measured relative to 
some suitably chosen angle of convergence. The central difficulty In 
defining uhat is meant by the process of extracting binocular disparity 
fr-ofc an image is that disparily he* to r*fSr to a physical entity - a 
point oo a visible surface - yet it appears that uo coNpute it at a level 
far below that at uhich the world is described in terms of surfaoos and 
Object a IJulesz 1971). ] t uas probably this fact that made ao surprising 

Julesz's conclusion that disparity aseignnsn-t is a lou-lev^l compy191>00# 

]n order to Cbopute disparity correctly. the fpi I suing stops »usl 
be carried euti firstly, a particular location on a surtae* ii - ! the Scans 
^ust be located in one Image; secondly. the identical location must be 
Identified in the other image; thirdly, tha relative positions of (h<e 
tuo images of that location Host be measured. The most Interesting and 
most trout) I coo rg part of the process concerns thi selection of a location 
on the viewed surface, and the identification at its tub Images. The 
■difficulty is that the choice of a point on the surface must t)« made frtrtl 
the lmagest if it could he chosen in same absolute way - by lighting it 
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up at that point for oHairple - the problem would he simple* 

Ue ore now in a position to understand why (.ha disparity 
campvlst ion is not the same thing .is a grey-1 eve 1 cqrreiat ion* The reason 
is that grey-level ftcasurensnts correspond to properties of tha jjeage. 
rather than to properties of the objects being viewed. An f» r y> co¬ 
ord inate pair on an image is an artefact Of the transducer, a Inca It dees 
not define a point on a physical surface in a way that allows it to &S 
identified on the other image. The mog,| glaring example of the failure 
io the case where an imago point corresponds !o two surface poinds, the 
nearer of which is transparent or transl ueentl a goldfish in a pond ie 
one such case, where the water surface ard the goldfish are 
simultaneously via idle. Other swamp lea are provided by figures 5.7*1 and 
Of Juleaz (19711, But the argument applies equally well to the 
case of a single visible surface, and its consequence i? that grey-level 
.■.etching methods are Incorrect. On simple images, the method will 
succeed, because it Is close enough to the right Idea; but as Hqr 1 «| al. 
(19731 have found, it nil I not succeed on conples images because it ia 
based on incorrect premise, Thair technique of introducing local 
smearing may ha viewed as a way of beginning to Identify 3 point in the 
image with a point on the physical Surface (by adding additional 
constraints on what is matched!! in bo far as it does do, the method will 
become more raarable, Put it Is probably better to altack the underlying 
■ssue directly. 
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The use of Ijiu - l.c-vn | symbols 

Ira order to formulate thi? disparity computation in a usable way, 
us therefore need to tie able to identify surface points from the tuo 
ImegESj and match them uo. It is clearly fruitless to try to I abfl I points 
of a evaoth featureless Eurface* but if that surface contains a scratch, 
boundary* or ether identifying physical mark which produces a local and 
fairly sharp change in reflectance, that change in refinance may be 
used to define the surface point* Provided that the Change In reflectance 
ha-3 been identified and described separately in the tho images, the 
resulting descriptions will correspond to an underlying physical reality* 
7hr* computation of such a Iou--level description from one image hag been 
dealt with at length elsewhere- (flarr 1974a S 1374b), and ii called the 
IbU-lcvel synodic representaticn of an image. Hence m3 see that provided 
Stereo matching takes place between tuo low-level symbol ic descr ipt i one. 
it is -a well-founded opera! ion. 

Finally* ue need to ask whether any reasengply cokplek 
■easurement Could be used - or i g the-r@ something special about a Iwi- 
level gymbuI 1 e description? SiHple 0*11-1 ika measurements are for e»tg«ple 
nearly guit able* because they are sometimes quite near to low-level 
assertion*; but when assigning disparity values to ainple cel Ip, one 
■tcct-e all the usual problems associated ui|h measurementg - a uhple git 
Of Simple cel I a, at noighpeuring positions and bri B ntations, corresponds 
to the underlying scratch or whatever I h the image* and IMe that 
compien which ng-eds to be Hatched against the corresponding conple* 
dmcrvod f-r-um the other image. If the important Hatching step Is carried 


FIGURE 1 


Figure K L denotes the collection of left-image symbols, and R the collection of 
right-image symbols. These are connected through the set C of disparity value 
symbols. The sets DL and [>r referred to in the text are shewn fri the figure 
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out on noth individual siiiple cell measyrHjiisnt, the campy tat ion become-# 
very uneconomic |, Hence one may expect that whsn disparity ia actually, 
agsigned, the procEas operates on a very I on- 1 eve I symbolic description* 
This method niff fall only when tha low-1 eve I deer ipt ion? obtained from 
the- two images are very different but this ia comparat i va I y rare, and 
O-ne seems to notice it when it happens, In any case, th-ls circumstance 
cannot be dealt with at the same very low iewsL 

The problem has ecu been reduced to the comparason of top leu- 
level symbolic description#, and ttiE assignment of disparity values ta 
paira of symbol#, drawn approprlately from e-3Ch image. He turn now to 
examine briefly the rylos and const rain ta to which this process ia 
subject* 

The "y,se once* 1 condi_tjon 

He have seen th^t 3h efehent of a low-1 eve I symbolic dsacr ipt i on 
of an image corresponds to a physically identifiable entity In a nay that 
an Image eo-drdinSte does not. and In which neaeurenents made on an image 
only approximate. This allows one to state the first condition that 
controls tha matching of two low-level symbol ic descr I pt i one, ]t ia that 
each low-level symbol should be -assigned exactly Oho disparity value., 
which in turn implies that it should be associated with at most one 
symbol computed from the other Image. This i a called the "use-once* 
condition, and it Is non-linear. 

The use-cnee condition may be Implemented So the following way* 
L&t L bo the set of all left-lnage low-level symbols, and Pet R Ds the 
set of all right-image low-lev#! symbols. To each element * in L+ there 
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carregpcktid sever 34 elements I ft R, ore far each of the pues ible disparity 
values; and for each element y infl, there is a corresponding set of 
elements In L» T h i s situation is. ■ I lustra ted in figure 1, The matching 
cf one element from U m th &n* from R corresponds to the aeBignmerit Df a 
Single disparity value to both elements, SO let us introduce a third sat 
D that eonsislg of collections of elements representing all of the 
possible disparity values that may he bound to each fow-lovel symbol. In 
principle, one needs one such collection for each Iom-IbvbI symbol, 
though memberg of t and ft share eieiients in D in the appropriate nay tssa 
beloul. The use^oncu condition translates Into the constraint that each 
element of L and of R nay be bound to at (Oil one element of 0. 

In practise, the set D wl i I be very large unless steps are tatter 
to economise on the number of units that are necessary to represent the 
disparity values* so tef gs consider how disparity representing units may 
bfr shared between several elements Df l Isayl, 0 has to bB large enough 
SO that tal each I om- love-1 -symbol can find an unuoed collection In 0 that 
can be uBed for representing i te disparity; and [p| the cores spends noe 
Setueen L and A through t) it ms M -rfaf rnad. To accomplish this, the 
collection of left- (or right-t inage symbol's that share a given 
d i spar i ty-reprfl3cn| i ng unit should have the property that it is very rare 
for tuo to bo provoked s i nu I taneously by the image, 

Uh^n one considers hou to construct a parallel netuork that 
implements the USe-dncs condition, it is apparent that at least three 
variables must be Pcconodated: aitfrn&ion and declination In the visual 
field, and disparity, ft satisfactory arrangement is shQMft In figure 21 In 
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Figure 2. One possible geometr ical arrangement of the get 0 of aymPple 
fur disparity values Is to arrange D in stripes &f constant disparity, 
the figure shous hou this may he done. The stripes running nearly 
herijontal ly contain units M i t h constant disparity dl« The x-coardinats 
in the image is represented by the H-coordinsts in the figure* and the y- 
Cbdrdinate rn the image corresponds roughly to the y-OOorJinate in the 
fi^jre. Ihs y-axig Is magnified in the figure because a given y 
coordinate in the image needs to Lie represen to if at each disparity, The 
point of the figure >9 to shou how low-level left- and right-i*sge 
eymhols share elements of □ . Ths sets Dl and Or* that are described In 
the tent and shoun in figure 1* are arranged to tress ono another, and 
d i v i de □ up in tuo different Mays, The connexions that implement the upp- 
once condition run along these tug diroctions, The connsKl&n* that 
implement the suggest ion interaction run mgrg nearly parallel to the 
stripes, and they join stripes Corresponding to the same disparity Cdi) 
in neighbouring positions in (ho visual field. The conne* I one that 
measure local goodnesS’Of-tit Mould have an undirected distriCution, 



FIGURE 2 
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thifl. the units represent intj dispari ty values on e- arranged in stripes of 
constant disparity. The collections in Q that represent disparity values 
for Iflti-rmayrj symbols fLI I s-B along thp tfiagonal Hnsa marked 01; and 
thee* fi>r right-imago symbols lie along the opposing 59 1 of lines Dr. 

Thug 0 divided up in two ways into disparity representing unite, which 
arc a >*ui! tansouSly shared in an appropriate faahron by L and R. The 
connoKipne that implement the uao-once condition are dearly larked; they 
run aiding 01 and Or, joining places that ecmiain reprosentat 1 one of all 
possible disparity values that could be bound to a given loft- or rlght- 
gemerated symbol. (In 9 neural inplenentation dF this schene, such 
connexions would be i nh i pi tory. I 

There are interest ing differences between' the I rrp I Icat ions for 
neurophysiology of theae ideas 4 and dF the node I of Jules*. firstly. the 
important part of the computation involves constraints on the (Usparlty 
values that may be bound to lew-1 eve I symbols. The magnets in Juleai's 
model seem however to correspond to rather unspecific local disparity 
values, and up saw earlier that the disparity computation cannot be 
accurately expressed In these Terns. The second point rests on the way 
in which disparity-repre?entIng sets in D are assigned to L and to R. The 
most economical way of forming thE col I act icing nf IcM-levsl symbol®. that 
are to use the same disparity-representing units. Is probably to group 
togothar all symbols that describe a small region and a small i orl eolation 
range. Only very rarely will two such symbols be used simultaneously. if 
some schetie of this sort were being used, it Mould account for the 
existence of cells that behave sensitively (o disparity, but are 
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relatively tolerant to position and orientation (Hubei ft Wiese I 13783. 
Furthermore, it Mould be within these units that the sain disparity 
computation is being, tarried out, and between which the govern I ng 
corttiBxions aheuld he made. One wold net expect to find other cells, 
expressing a free-floating diaparily value in a "region* of the image, 
because the essence of the computation requires that it bo carried Out on 
bindings to lctM*vel aymbole, The model of Ju| gBZ would, ] thin*, lead 
one to SKpect such cells. 


Qj_sparj_tL| jij. continuous almost ev erywher e 

The uoe-once Condition must da satiefied by the final asBi B nm e rit 
Of di spar fly values to the low-level symboFic dSecr ipt I Ofls, but it is not 
fliueh help in finding It, Mhen applied to a random dot stereogrsh, it 
U ' " C!nsure **“ dsscr i p t i on of each dot, or group of dot*, in one 

***** is mat ad with not If ore than one alii Ur description computed from 
the other image; and a solution that satisfies this anrf I gives very f E w 
dots out wilJ probably be correct (see the next condition}. But (here is 
another useful property of the real world that can be introduced with 
advantage to spoed the analysis. It is that except at object bound*- I e s. 
disparity is 3 fund Ion that varies snoothly over the inage. Fine 
texture, which is the best source of disparity information about a 
surface, »m be represented St the lowest level by assertion* about very 
smaM features; and except at object boundaries, the disparity values 
that become bound to neighbouring symbols will be about the same. This 
fact a Mows thfl existence- of an interaction that H propo so e “ the disparity 
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used at one point as a strong candidate for the value at neighbouring 
points: it correSpends In Julesz’s [1971} hodel to th.E lateral coupling 
provided by Ihr snail springs that join adjacent magnets* The 
implementation of this constraint in Sperling's U970J Model rft obocur*. 

The imp I ehentat i on of a suggestion Is one of those questions that 
it is not prof I table to pursue in detail* because of the difficulty in 
testing, either physiologicaIly or computatlong I t y, the conclusions to 
which one may be led* 3 shall therefore ma^e only three points about it. 
The first ig that, in principle, one would like a eugytstion to influence 
the route to a solution, without disturbing the values In the solution 
once they are found. This implies a time-dependence in the interact ion. 
Secondly, in- order that the ftolutiob hay be stable* one would probably 
also need to add a small DC component. The third point, and one which 
may actually be useful, Concerne the geometry of the suggestrOh 
i n terac t i ong* and this 15 shown in figure 2. There are eofimex 1 oh e between 
alt disparity units I hat represent similar disparity, values, and that 
rsfor to symbole in nearby portions of the visual field, in a neural 
implementation, they uoultf bo excitatory. 

toadnee_H -of- f i t 

The final important aspect of disparity measurement i• the 
question of hpu. oat I ef actor y a sc lot ion is, Juleoz emphasised the need 
for ouch a measure, and in his model, it corresponds to the total 
potential energy in the tuo t superimposed aseenPlies of magneto.. Sperling 
{19705 also used a potential energy measure rn his formulation, Juloez 
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showed that in an ambiguous stereogram, ue perceive the be t tsr-eorreI oted 
solution oven It tooth have quite hlph tor re I at I emt, This is good evidence 
that tne matching proceed Is parallel rather than serial, and that the 
qoodncag-of-f1r measure ie computed on a local basis. 

In an Implementation of the kind that we are discussing, the 
goodneso-of-fit of a solution nay he measured by th# proportion of left- 
and right-*-image symbols that become bound to disparity values, jn a 
perfect SO I ut inn, the proportion will hft l,0i and an inappropriate 
disparity binding <411 have the effect pf depressing the proper t ion of 
correct bindings in its neighhourhood. The local goodness-Of-fIt function 
would affect the conf idoneo with uhich disparity assignments are hade 
locally - Kb, the strength with uhich they are asserted - nhrch in torn 
would affect the potency uitti uhich they arg suggested to nearby regions* 
The goodness-Ot-f 11 function would therefore be implemented ty a unit 
that depressed the local dl soar i ty-represent log units by an amount that 
depended upon the proportion of image syiibala in the vicinity that have 
bEen assigned disparity values, I do not sea how to test for the presence 
Of auch unite, except by tri@1-and*error. 

Summary of the d i spa r i ty intera ct ion-s 

The interactions descrioed above are now dlraun together, and the 
conditions that are necessary to an i nplementat 1<Sh of this kind are made 
owplicit* 

IH The disparity assignment is made a<s a result of a notching operation 
performed on fcuo lou-level symho11c do scripti ons, CQNpuled Independently 
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from the left and right im-ages. 

12} The matching is i up Iemented by applying conditions to the process of 
binding syetroHe disparity deacr iptom to the iou-level symbol * 4 These 
constraints are the use-oncs condi tion, tha sug^ ca t i on interaction, and 
ma k 1 *m 3 i ng the goodness Of fit. 

53) The use-once condition requires interactions uhoae geometry appears 
In figora 2. These interactions inhibit the confidence of those 
90S i gn*ian t a that they connect. 

C4) The suggestion interact ions have the gaotrstryi shown also in figure 2. 
They connect disparity descriptors that represent similar disparity: 
valuta, 3nd that are capable of being hound lb low-ievei symbols 
referring to neighbouring positidhe in the visual field. Such 
interactions would probably have a tima-depantfsnit cohponent ee u«M as 3 
DU component, 

ISJ fla* i it i a i ng the poodneaa of fit of a solution would hav* to be 
i mp I e hen tad by a local goodnEsa-of-f i | function, that measures the 
proportion of low-level symbols that have auccssafuily been bound to 3 
disparity value, and ubich affects the confidence level of the bindings 
In that Jocal reyichK 


Dis cussio n 

[ shoi I not attempt in thia note to derive ang properties of tf»e 
above system. E have bean unable to Hake much progress with an analytical 
approach to tha prableti, and the asounf of time required for a 
computational study is very large. The approach eat out hare doee however 
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illustrate [al hou the disparity ecmputatI cm may be us II-f ounded; fb) the 
importance Of I ou- 1 eve I symbol* in the formula (lent and He* hey the 
i nspbrt-anit constraints tray at least in a general uay be represented by 
connexions ui th a a traightf oruard geonelry. This kind of gaome ir j cal 
arrangement i$ one that it is becoming possible to detect. 

The second large issue concerns tha way In which disparity 
information may be used, It is one thing, to assign disparity values to 
Id«-level symbols, and quite another to divide up an image into regions 
on the baei* of disparity information aTone* and compute a description of 
the spatial ok tent of each region. For example, in gny stereogram, the 
orientation information associated urth fh« email equprs* or groups of 
square a that are actuary Hatched bears no relation (o tha orientation of 
(he edge at ubich disparity changes. One way of computing the higher* 
induced edges would of course be to treat disparity like Intensity, and 
subject i I * values to a process like that used to obtain a fou-level 
symbolic description from an Intensity array (Harr . This eothod 

Seems somewhat clumsy, houevar, because disparity is n&( the on Iy kInd of 
in-formation {bKcludiny Intensity! frpn which directions and boundaries 
may be computed locally. Texture ehangse are another s^anpi*, and to are 
more abstract out Fines, like the envelops of a sparse tree in winter or 
the boundary defined by a row of grnsM. separated bushes across & garden* 
On® would like to know whether g 11 &f these problems may be dealt with by 
a single method that can describe configuration* pf "place*" In an image 
- these pi acts being identified by a rather simple Kind of Focal 
measurement »ade on the relevant typ& of i nf or mat I Ob, Thera seeing to be 
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cl Ear evidence [and a definite computational hocdl for ouch a me chan i Bin “ 
ar& pf its *3 in functions being to set up an orientation in the image at 
a point, to describe conf i yurat iono of places relative to that 
or ientat i on* and (a influence the direction relative to which local 
shapes in an i mayo aro rf o sdr i bed. It is however far from clear whether 
one such mechanisr would suffice to service all Of the demands of this 
kind, or whether the slightly differing totipu1311ona3 rsquirsmente force 
the a>sistance of a neither of separata, but similar mechaniems. The 
question will he raised elswhera [Harr i37SI. 

Af know I cdgersn 13 Work reported herein was conducted at th# Artificial 1 
Intelligence Laboratory, a Hass&chu e stts Institute of Technology research 
program supported In part by (he Advanced Research Projects Agency of the 
department of Defense and monitored Uy the Dffico of Hava I tte search under 
Contract number frffi00l4-?0-A-@362-0305. 
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